Focusing on the problem of robust video object tracking, a robust generative algorithm based on sparse representation was proposed. Firstly, object and background templates were constructed by extracting the image features, and sufficient candidates were acquired by using random sampling method at each frame. Secondly, the sparse coefficient vector was got to structure the similarity map by an innovative optimization formulation named multitask reverse sparse representation formulation, which searched multiple subsets from the whole candidate set to simultaneously reconstruct multiple templates with minimum error. Here a customized Augmented Lagrange Multiplier (ALM) method was derived for solving the L ₁-min problem within several iterations. Finally, the additive pooling was proposed to extract discriminative information in the similarity map for effectively selecting the best candidate which the most similar to the object template and was most different to the background template to be the tracking result, and the tracking was implemented within the Bayesian filtering framework. Moreover, a simple but effective update mechanism was made to update object and background templates so as to handle the object appearance variation caused by illumination change, occlusion, background clutter and motion blur. Compared with the other tracking algorithms, both qualitative and quantitative evaluations on a variety of challenging sequences demonstrate that the tracking accuracy and stability of the proposed algorithm has improved and the proposed algorithm can effectively solve target tracking problem in these scenes of illumination and scale changing, occlusion, complex background, and so on.